Identification for Sources

The classical transmission problem deals with the question how many possible messages can we transmit over a noisy channel? Transmission means there is an answer to the question “What is the actual message?” In the identification problem we deal with the question how many possible messages the receiver of a noisy channel can identify? Identification means there is an answer to the question “Is the actual message u?” Here u can be any member of the set of possible messages. Allowing randomized encoding the optimal code size grows double exponentially in the blocklength and somewhat surprisingly the second order capacity equals Shannon’s first order transmission capacity (see [3]). Thus Shannon’s Channel Coding Theorem for Transmission is paralleled by a Channel Coding Theorem for Identification. It seems natural to look for such a parallel for sources, in particular for noiseless coding. This was suggested by Ahlswede in [4]. Let (U , P ) be a source, where U = {1, 2, . . . , N}, P = (P1, . . . , PN ), and let C = {c1, . . . , cN} be a binary prefix code (PC) for this source with ‖cu‖ as length of cu. Introduce the RV U with Prob (U = u) = pu for u = 1, 2, . . . , N and the RV C with C = cu = (cu1 , cu2 , . . . , cu‖cu‖) if U = u. We use the PC for noiseless identification, that is user u wants to know whether the source output equals u, that is, whether C equals cu or not. He iteratively checks whether C = (C1, C2, . . . ) coincides with cu in the first, second, etc. letter and stops when the first different letter occurs or when C = cu. What is the expected number LC(P, u) of checkings? In order to calculate this quantity we introduce for the binary tree TC , whose leaves are the codewords c1, . . . , cN , the sets of leaves Cik(1 ≤ i ≤ N ; 1 ≤ k), where Cik = {c ∈ C : c coincides with ci exactly until the k’th letter of ci}. If C takes a value in Cuk, 0 ≤ k ≤ ‖cu‖− 1, the answers are k times “Yes” and 1 time “No”. For C = cu the answers are ‖cu‖ times “Yes”. Thus LC(P, u) = ∑‖cu‖−1 k=0 P (C ∈ Cuk)(k + 1) + ‖cu‖Pu. 1